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ABSTRACT 

Advancements in technology and innovations in education allow universities to entertain new ways of teaching and 
learning. Some views of what higher education should look like today include that it be easily accessed by anyone who 
wants to be educated, that it cost less than it currently does, and that there be a significant increase in student engagement, 
experience, and the quality of education. This paper presents quasi-experimental data of how various online tools and 
teaching strategies impact student learning outcomes, satisfaction and engagement. Specific variables impacting social 
presence, affect, etc., where tested to determine their impact on different student outcomes such as grades, feelings of 
isolation, student engagement, and perceived authenticity of course materials. Findings suggest that, despite the literature, 
only some factors had a significant impact on student outcomes and that while some outcomes transferred well online, 
others did not; particularly, peer activities and participation in some course components were hindered online. Considered 
here are students’ experiences with online learning, including hybrid and inverted courses, and teaching strategies that 
help meet challenges in different higher-education learning contexts. 
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1. INTRODUCTION 

Advancements in technology and innovations in education allow universities around the world to think up 
new ways of teaching and learning that can sometimes help instructors avoid limitations experienced in 
traditional models of education. Technology drives and enables a lot of the new and different methods of 
online teaching and innovation that we hear so much about in the media, amongst our colleagues, and across 
institutions. Some views of what higher education should look like today include that it be easily accessed 
by anyone who wants to be educated, that it cost less than it currently does, and that there be a significant 
increase in student engagement, experience, and the quality of education. Whether for online courses, hybrid 
courses, or some other learning context, many instructors are wary of using too much technology since it can 
be distracting and they worry that some educational technologies can take away from their teaching and 
students’ learning experience. There is a good argument that one can use technology in ways that brings 
classes together; so for example, using technologies and devices that students are using anyways but embrace 
them in ways that create meaningful interactions rather than distractions, or using them in ways that empower 
both instructors and students, and engages them online or in and out of the classroom. 

It seems obvious that online learning technologies help instructors innovative, but what’s the evidence 
that online learning, or the tools and teaching methods facilitate learning? Presented here is Part-Two of a 
three part quasi -experiment that considers teaching strategies and educational technologies that push learning 
beyond boundaries often found in traditional teaching models. Boundaries considered here include social 
presence, affect, behaviour and cognition. Part-One of this research series, has been published elsewhere 
(Berry & Kushnir, 2013). Data in Part-One compared face-to-face and online teaching and learning; Part- 
Two, adds new data on teaching approaches, strategies and course/curriculum design that focus only on 
online teaching and learning. 

In Part-One, half of the students completed an Introductory Psychology course in a traditional face-to- 
face setting while the other half completed the same course completely online (with the exception of term 
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tests and the final exam to ensure academic integrity). In Part-Two, all students completed the same 
Introductory Psychology course entirely online (again with the exception of term tests and the final exam). 
Added here are the findings from the two additional quasi -experimental groups of online students. One of 
the two groups of students received 30-60 minute lecture videos as part of the online course materials, while 
the other group received the same lecture videos chunked into 5-15 minute lecture video segments with 
embedded quizzes that popped up during the short lecture clips. Specific variables impacting social presence, 
affect, etc., where tested to determine their impact on student outcomes (e.g., grades, feelings of isolation, 
student engagement, feeling like the course materials were authentic, etc.). Results show that, despite the 
literature, only some factors had a significant impact on student outcomes. This research series helps 
researchers, instructors and other education specialists understand factors that impact online teaching and 
learning. It also contributes to our understanding of how specific teaching strategies can impact online 
learning in different higher-ed contexts such as online, inverted, and hybrid courses. 


2. LITERATURE REVIEW 

Enrollment in online education continues to grow at a quicker pace than enrollments overall in higher-ed. As 
many universities and colleges struggle with issues of space, scheduling conflicts and budget cuts, some 
believe that online education offers cost effective alternatives to traditional classroom teaching. In 2010 the 
Sloan Consortium reported that online enrollments were up 17%, compared to 12% the previous year. In 
2011, at least 33% of college students had participated in at least one online course and the majority of these 
students (over 82%) were undergraduates (Allen & Seaman, 2011; Parry, 2010; Salcedo, 2010). 

Online courses can be convenient in higher-ed, mitigating constraints of time and space in traditional 
face-to-face courses, and allowing institutions to offer more courses and effectively meet the growing and 
changing needs of students (Gould, 2003; Macon, 2011). The literature is positive for the most part; some 
argue that it might be more cost effective for institutions to offer online courses since they often require less 
overhead than physical classrooms. Others argue that online courses expand the reach of the institution, 
potentially attracting international students and increasing revenue. Some consider online courses as a way of 
retaining undergraduates and ensuring that they graduate on time, while others see the benefits of allowing 
students to learn at their own pace, the flexibility around studying and working (either part-time or full-time), 
the related savings on commuting, of childcare, etc., (Lei & Govra, 2010; Macon, 2011; Salcedo, 2010; 
Wuensch, Aziz, Ozan, Kishore & Tabrizi, 2008). 

Online learning is not always viewed positively. Some curriculum committees routinely question the 
academic rigor of online courses, express concern about academic integrity, and worry that academic rigor is 
often compromised to facilitate online delivery (Schoenfeld-Tacher, McConnell, & Graham, 2001). Before 
institutions invest in online courses, it is important that there be added value, a positive impact on student 
learning and engagement, and assurances that rigor and academic integrity are maintained. 

Some authors argue that online courses increase the workload of faculty. According to Pallof and Pratt 
(2007), it can take 2 to 3 times more time to prepare and deliver the curriculum for online teaching compared 
to face-to-face teaching. Most of this time is due to the effort required to develop online materials (e.g., 
record/edit lecture videos), manage and upload the resulting large files. Despite the extra work, over 30% of 
faculty report teaching online and this is reportedly increasing (Simson et al., 2006). 

Consistent with Means et al., (2009), for this study, online learning is defined as learning that takes place 
partially or entirely over the internet. This definition excludes purely print-based correspondence education, 
broadcast television or radio, video conferencing, video cassettes, and standalone educational software 
programs that do not have a significant internet based instructional component. 

2.1 Student Engagement and Interaction in Online Courses 

While the literature is generally positive about online interactions, and some authors reporting high levels of 
student engagement and interaction (Schoenfeld-Tacher, McConnell, & Graham, 2001), classroom 
interactions and feelings of community are often the reported benefits of face-to-face courses (Homberg- 
Wright &Wright, 2012). When considering the types of interactions (e.g., using Blooms Taxonomy), some 
report high-level interactions in online instruction compared to face-to-face instruction (Schoenfeld-Tacher et 
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al., 2001). Some students rate interactions online to be about the same as those face-to-face (Allen & 
Seaman, 2011), but students who prefer face-to-face classes, do so because they enjoy the classroom 
interactions (Daymont & Blau, 2008). Quality online courses must incorporate a substantial amount of varied 
interactions (Clark-Ibanez & Scott 2008) and the level of interaction in online courses is often seen as a 
predictor of students’ perceived learning (Rovai & Barnum, 2003). 

2.2 Student Satisfaction and Social Presence in Online Courses 

Student satisfaction in a course is often seen as an indicator of successful learning (Parkhurst et al., 2008; 
York, 2008). The argument being that there is a relationship between student satisfaction and students’ 
perception of the quality of their learning (Piccoli, Ahmad, & Ives, 2001). Substantial and timely interactions 
between the students and instructors can reflect high levels of student satisfaction, and this high level of 
satisfaction can also indicate that teaching methods strongly reflect learning goals and student expectations 
(Moore, 2005). However, the link between student satisfaction and student learning is not clear since students 
might report that they are more satisfied with a course that they perceive as being easy, fun, or less 
demanding; these attributes may not necessarily be linked to real measure of success. Macon (2011) reported 
that undergraduates tend to be more satisfied with face-to-face courses than with online courses, while York 
(2008) found that students were as satisfied with online course work as with face-to-face course work; Fillion 
et al., (2007) also found online students to be more satisfied than face-to-face students. Despite these 
competing findings, it is reasonable to expect that in a classroom where successful learning is evident, then 
student satisfaction will be higher (Driscoll et al., 2012). 

Online students who are more satisfied with their learning experience and satisfied with their instructors 
have been found to have a greater sense of social presence in the course (Lyons, Reysen & Pierce, 2012; 
Richardson & Swan, 2003). There is lots of research that suggests that it is difficult to develop social 
presence online at a level that satisfies students. Instructors have to work hard at creating an online presence 
(Richardson & Swan, 2003; Salcedo, 2010). 

2.3 Student Learning Outcomes in Online Courses 

For the most part, when grades are considered, the literature reports no statistically significant differences in 
learning outcomes between online and face-to-face students; some authors report that online students slightly 
outperform their face-to-face peer in the same course (Beeckman et al., 2008; Beyea et al., 2008; Lim et al., 
2008; Parkhurst et al., 2008; Salcedo, 2010). Few show significantly higher differences in learning outcomes 
for online students, and as mentioned above, academic integrity is often a serious concern (Schoenfeld- 
Tacher et al., 2001). Some authors have found that online learning is at least as effective and robust as face- 
to-face learning (Brownstein et al., 2008), providing the same level of instruction (Carter, Emerson, 2012; 
Driscoll et al., 2010; Russell, 1999). In some cases, online instruction is reported to be less effective than 
face-to-face instruction (Urtel, 2008) with students having difficulty keeping up with the requirements of the 
course (Keramidas, 2012). Also, some authors report evidence that hybrid instruction (i.e., combining online 
and face-to-face elements) has a greater advantage relative to purely face-to-face or purely online instruction 
(Means et al., 2010). 


3. RESEARCH RATIONALE 

This study evaluated the impact of various teaching strategies and online tools (e.g., the use of lecture videos, 
quizzes with rapid feedback, peer-to-peer activities, and online assignments) on the following factors; 

1 . Student engagement, interaction and feelings of isolation 

2. Satisfaction of learning experience, social presence (and how well students felt they got to know the 
instructor), and authenticity of course materials 

3. Student learning outcomes 

We set out to investigate what are students’ experiences online, if there are variables or teaching 
strategies that impact the factors listed above, and if certain strategies correlate to better outcomes. 


135 



ISBN: 978-989-8704-08-5 © 2014 


4. DESIGN OF THE STUDY 

4.1 Participants and Description of the Study 

A total of 60 students enrolled in an introductory psychology course at an urban university participated in 
Part-One of this study. About half of those participants were enrolled in a face-to-face section (n=31; section 
1, Part-One ) and the other half in an online section (n=29; section 2, Part-One). The students were self- 
selected and chose in which section of the course they enrolled; they were only restricted from switching 
between sections after final date to register in the course (a date set by the University Registrar’s Office). A 
total of 52 students participated in Part-Two of this research series. Similar to the first, about half of the 
participants were enrolled in one section (n=25, students received 30-60 minute lecture videos, and all course 
messages delivered using the text announcement tool in the institution’s learning management system; 
section 1, Part-Two ) and the other half of students were in the other section (n=27, students received 5-15 
minute lecture clips with embedded videos, and course messages delivered via a video messaging tool to 
increase social presence; section 2, Part-Two). As was the case in Part-One, students were self-selected and 
only restricted from switching sections after the final registration date (though students in this second part of 
the study had no reason to move between sections since they were unaware of any explicit differences 
between the groups; as far as they were concerned there were just two online sections of the course available 
concurrently). Across all groups, there were no significant differences between first year university average, 
cumulative average, and high school entrance average. All online students received a “presence” video from 
the instructor as well as all of the lecture material online. Students came to class (physically) only 3 times to 
complete the requirements of the course (for two term tests and a final exam). The face-to-face group (section 
1, Part-One) met for 3 hours, 2 times per week for lectures during a compressed summer semester of 6 
weeks. A key component in the face-face section was peer activities, which were facilitated with student 
response system (clickers). This particular course component was replicated in the online groups by using 
online quizzes and a discussion board in the institution’s learning management system. Initially, break out 
rooms in Adobe Connect were used to facilitate synchronous peer activities but this had to be abandoned due 
to a problem with the tool and therefore Adobe Connect was only used for synchronous online office hours. 
Students received the same course content and course components in all sections of the courses delivered by 
the same instructor (i.e., Summer 2012 and Summer 2013). 

Online student experience was assessed using a half-way checking in survey, while all students received 
an end-of-term survey and other learning outcome comparisons included 6 quizzes worth 10% of the final 
grade, 2 term tests each worth 20% of the final grade, a final exam worth 30%, an assignment worth 10%, 
peer activities and participation worth 10%, and overall final course grades for all groups were compared). 

4.2 Analyses 

Analysis of the data included independent /-Tests to measure any differences between final grades for the 
online and face-to-face students in Part-One, and between the two online groups in Part-Two. ANOVAs 
were calculated to measure any differences between all course grades across the four groups in both parts of 
the research series. Qualitative analyses of the open-ended survey questions included response frequencies 
of the survey questions across the groups and weighted word lists that were calculated and puzzled out into a 
word clouds that were generated from the students’ text answers. The word clouds represented a summary of 
the text that students wrote in their open-ended answers. A user generated word cloud visualizes information 
that is related to a specific survey question and, in essence, it depicts visually, the frequency of specific topics 
that students write about in their answers. The importance (or frequency) of specific words is displayed using 
font size (as in the example below), font colour, or some other attribute (see Bateman et al., 2008 for an 
overview of word/tag clouds). 


5. RESULTS AND DISCUSSION 

5.1 Student Engagement, Interaction and Feelings of Isolation 

Class interactions and feelings of community are some of the reported benefits of face-to-face courses 
(Homberg-Wright &Wright, 2012) and in some cases, students who prefer the face-to-face classes do so 
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because they like interacting with the instructor and their classmates (Daymont & Blau, 2008). This suggests 
that online environments should provide lots of opportunities for engagement and interaction. In our study, 
students were asked to indicate in which part of the course they felt most engaged; as indicated in Figure 1, 
students reported that the peer activities and the Science Meets Art project (a course assignment that was 
shared amongst classmates) provided the most engagement. The demonstration videos as well as the lecture 
podcasts also contributed to student engagement. Online students were also asked whether they wanted more 
interactions with their classmates, and if so, what sort of interactions they wanted. Interestingly, the request 
for face-to-face interactions and study groups came up frequently in students’ responses (Figure 2). This 
provides support for a flipped or inverted classroom where the lectures and usual in class material are placed 
online for students to access outside of class, and where class time can be used for activities that would 
normally be done at home such as homework and assignments. This gives students and instructors the 
opportunity to have engaging, interactive sessions such as collaborative work and in-class activities that 
focus on higher level cognitive activities (Bull el al., 2012; Brunsell & Horejsi, 2013; Milman, 2012). 
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Figure 1 . What part of the course or course activities helped students to feel most engaged 
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Figure 2. Students indicated that they would have liked more interactions with classmates; 
these are the sorts of interactions that they wanted to have (online students only) 

When asked if they felt isolated, online students in section 2, Part-Two (the group with 5-15 minute 
lecture clips with embedded videos, and course messages delivered via a video messaging tool to increase 
social presence) reported the least isolation (see Figure 3). This suggests that these students feel more 
included and connected in the course compared to the other online groups who seem lonelier. 

Did you feel isolated in the course? 
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Figure 3. Online students’ report of feeling isolated in the course 


5.2 Student Satisfaction, Social Presence and Authenticity 

We also asked students about what they found most satisfying about the course. Overwhelming, across all 
groups students reported being satisfied with their experience and their learning; figure 4 shows what 
students found most satisfying. As reported earlier, online students who are more satisfied with their learning 
experience and satisfied with their instructors have been found to have a greater sense of social presence in a 
course (Lyons et al., 2012; Richardson & Swan, 2003) indicated here (in Figures 5 & 6) by how well they got 
know the instructor, and what course factors helped them feel like they knew the instructor (Figure 7). We 
also found that this had an impact on students’ perceived authenticity of the course materials, and how real or 
artificial the course felt to students (Figure 8). 
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5.3 Learning Outcomes 


Students were asked what aspects of the course contributed most to their learning (Figure 9). While students 
reported that the interactive and active components contributed most to their learning, there were no 
significant differences between the groups on their grades for any of the course component ( i.e 6 quizzes, 2 
term tests, final exam, course assignment, peer activities, course participation, or final course grade). 
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Figure 9. What students believed helped most in their learning 
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6. CONCLUSION AND RECOMMENDATIONS 

We set out to evaluate the impact of various online tools and teaching strategies (podcasts, online 
assignments, quizzes, rapid feedback, discussions, and peer activities) on student learning outcomes 
engagement and satisfaction of their learning experience. We discovered that students had similar 
experiences across the different groups supporting the argument that online instruction can provide at least 
the same level of instruction and satisfaction as face instruction (Driscoll et al., 2010; Russell, 1999). In our 
study, we discovered that the teaching strategies that we chose actually influenced the teaching tools, which 
in turn, had some influence on our strategies. This is important that online courses be built on sound 
pedagogical principles in order to facilitate meaningful and successful learning. We took the necessary steps 
to design the online course around our teaching and learning goals that supported instructional needs and 
student learning outcomes, and not around the teaching tools that happen to be available. As more and more 
university instructors look to educational media and technologies to help engage students and enrich learning 
environments in different learning contexts, it will be helpful if future research focuses on the use of different 
learning contexts and online innovations to facilitate different teaching and learning methods, both in and out 
of the classroom. Advancements in technology and innovations in education allow universities to think up 
new ways of teaching and learning. It is in understanding the pedagogy behind the technology that will get 
us further along in understanding how to best implement educational media and technologies to enrich these 
learning environments. 
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